Distilling Knowledge from Well-Informed Soft Labels for Neural Relation Extraction
نویسندگان
چکیده
منابع مشابه
Distilling Knowledge from Ensembles of Neural Networks for Speech Recognition
Speech recognition systems that combine multiple types of acoustic models have been shown to outperform single-model systems. However, such systems can be complex to implement and too resource-intensive to use in production. This paper describes how to use knowledge distillation to combine acoustic models in a way that has the best of many worlds: It improves recognition accuracy significantly,...
متن کاملDistilling the Knowledge in a Neural Network
A very simple way to improve the performance of almost any machine learning algorithm is to train many different models on the same data and then to average their predictions [3]. Unfortunately, making predictions using a whole ensemble of models is cumbersome and may be too computationally expensive to allow deployment to a large number of users, especially if the individual models are large n...
متن کاملKnowledge Extraction from the Neural ‘Black Box’ in Ecological Monitoring
Phytoplankton biomass within the Saginaw Bay ecosystem (Lake Huron, Michigan, USA) was characterized as a function of select physical/chemical indicators. The complexity and variability of ecological systems typically make it difficult to model the influences of anthropogenic stressors and/or natural disturbances. Here, Artificial Neural Networks (ANNs) were developed to model chlorophyll a con...
متن کاملKnowledge Extraction from Neural Networks
0-7803-7906-3/03/$17.00 02003 IEEE. 1909 AbstractIn the past, neural networks have been viewed as classification and regression systems whose internal representations were incomprehensible. It is now becoming apparent that algorithms can be designed which extract comprehensible representations from trained neural networks, enabling them to be used for data mining, i.e. the discovery and explana...
متن کاملDistilling a Neural Network Into a Soft Decision Tree
Deep neural networks have proved to be a very effective way to perform classification tasks. They excel when the input data is high dimensional, the relationship between the input and the output is complicated, and the number of labeled training examples is large [Szegedy et al., 2015, Wu et al., 2016, Jozefowicz et al., 2016, Graves et al., 2013]. But it is hard to explain why a learned networ...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Proceedings of the AAAI Conference on Artificial Intelligence
سال: 2020
ISSN: 2374-3468,2159-5399
DOI: 10.1609/aaai.v34i05.6509